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/ FOREWORD 

« * • 

The National' Center for Research in Vocational" Education has worked 
with the state research coordinating units (RCUs) on cooperative endeavors 
since 1967 when development began on a nationwide information services 

ake A - to - E RIC (E dttea^i<mylr-Resot^ , — o^e 
example of this cooperation was a seven-state study of target audience 
needs which provided findings that later influenced the development of 
information databases, information analysis products, and information 
services. : J 

* * '* 

, The report describes the continuing joint efforts of several RCUs 
and the National Center to d.eveldp multi-state, multi-year programs of 
study. Such programs promise achievements which could not be reached by 
an ipdividual state, mor in one .year. The RCUs involved and the National 
Center are concerned with the problem areas as significant issues ia each 
state and throughout the nation. , 

This report .was prepared by Norman M. ' Singer with the cooperation 
of the National Research Cpordinating Unit Association* its Executive 
Committee, and the Research Coordinating Units participating. 

Robert E. Taylor 
Executive Director . . 
v National Center for Research < 

. m in Vocational Education 
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EXECUTIVE SUMMARY 



. The.NationaX^Cenfer for Research in .Vocational Education has continued to 
serve a facilitative role after initiating the planning by state research 



- ^Aaniixiig bLcne researcn 

-coordi n ating units (RCUs) for co/labora^ive studies in' three problem areas - , - 
youth unemployment, program effectiveness, and access for adults. The objective 
of this planning jstforj:^ was to* develop a plan, which projected the use of the 
combined talents funding, and in-kind resources of RCUs to accomplish research 
which could not be undertaken by- one one state or in one -year. ' • 

The RCU- Ek^cutive Committee developed initial plans for muJLti-state, multi- 
year m programs of sfudy and sought self- nomination by state RCUs for participation" 
in the programs. Participating states met with the National Center staff to' 
.plan a series of projects in jach program \area. "Representatives of planning 
■groups for each problem area presented a description of the study program ' V 
plan to other RCU personnel at the annual conference of the National Reseat " 
Coordinating Unit Association. The National Center staff Continued to meet • *• 
with participants in the program and has provided particular technical -assistance" 
to the several states and RCUs involved in the major study of vocational education 
program effectiveness. , , 

' V '- 

During 1981 \the National Center conducted further planning meetings, 
arranged and coordinated multi-state telephone Conferences, cooperated in 
joint reviews.pf a request for proposal released by one cooperating RCU 
reviewed the proposals submitted in response, and linked germane information.' ' 
resources from the National Center and other agencies to technicians developing » ' 
instruments for ttie effectiveness study. t • . 

Reports on the interim status of the study of vocational education effective- 
ness and the study of alternative delivery approaches have been planned' as 4 
presentations during the Seventeenth Annual National Research 'Coordinating N " 
Unit Association Conference on March, '30 through April 2, 1982 ^ ■ « 

* \ 

The cooperative studies conducted by RCUs and facilitated by the National* 
Center eventually will demonstrate the feasibility of voluntary collaboration 
among states on program improvement problems. This approach promises a 
viable and precedent setting course of action for solving problems of national 
significance in vocational education,. " , , 
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INTRODUCTION 

years, fhe National Center, has been responsible" 
to„the st^ate research coordinating tmits (RCUs) . 

/erall liaison ^between the' 



The responsibility has been included. in the overa„ 

National Cegter^ Dissemination and'Utilization Program anfl Se Research « 
fcoordinatlng^unirs through their Executive Committee. To a great, extent 
the Natiofaal' Center has collaborated with the r esearch 'coordinating u nits ^Jt ^L 
in a vafiety-of ways and has provided leadershij^to .them by facilitating ^'™ W T 



and coordinating the. sharing of information,^ joint planning among RCU 
and National Renter staff, by exchangin g< ^hnipal data (especially regarding 
the study of dissemination impact), by W*eQf9 around dissemination issues, 
;,by sponsoring the Executive Committee's planning meetings- conducted at the 
National Center, and by providing ^rms of training and assistance in areas 
of major concern. , ~ . - 

The Executive Committed for the National Research Coordinating Unit 
Association and the National Center staff have engaged in discussions* ° 
which" have illuminated the need for individual research coordinating finits 
% to share the^ efforts iii researching major proj/lems in vocational education/ 
RCU directors as. well as National Center staff recognized that €he shared 

''investigation of common problem areas could result in much greater cost- 
effectiveness, stronger research results', ^nd considerable* change in the" 
level of collegialHntersction among the research coordinating units and 

' the National Center for. Research in Vocational Education. The Executive 
Committee of the "National Research Coordinating Unit Association welcomed 
the .suggestion ^ that the National Center staff woufd 

. serve to promote and facilitate .collaborative research and^to proceed with 
joint planning for that purpose., * 4, * \ * 

, Three problems of major concern to* vocational educators were identified 
at an initial planning session described «in thi^ report. This ear.ly session • 
projected an important research collegiality among the-RCUs and the] National 



Center. 
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^ *' " .CHRONOLOGY OF ACTIVITIES . ' 

* * . • + f 

* . ; • • . i 

* The National Center for Research in Vocational Education has served * 
, in an initial catalytic roie by capacitating the planning among state \ 

Research coordinating units (RCUs)jfor collaborative studies, and in a 

Subsequent f^cilitative rtrle in -support of th#ir work-; — fhi^— fare- - include d 

C&SfttUg \ ttje NRCUA Executive Committee to develop general plans for 

collaborative studies, convening RCU personnel from states interested in t 
collaborating on studies in three problem areas* providing inf oitifetibn and 
, other supports for planning sessions, arranging and coordinating telephone 
• * \ conferences. among states,' and providing ^technical 'assistance* to' one of the , ♦ 
'collaborative studies. The National Center has facilitated^ communications 
"and planning amo^^states at diverse meetings and events related. tQ the 
collaborative studies. ' t ' »^ 

4 • „ • * • 

. . ' * K 

*It was deemed appropriate to^ organize this second report like the first-' 
% 'i as 4&chronology of activities related to promoting collaborative studie^s by 
i RCUs. ,This report represents a continuation of the^ea^lier report ^and ( 'vlike 
'the first, does not seek to describe the actual progress g>n the substantiye, 
problem areas. Substantive progress will be reported more appropriately 
through other channels by the various projects as they are conducted and 
% , eventually completed iinder sponsorship of RCUs. 

* \ * * " 

This report provides a brief chronicle to summatize the significatk 
developments which occurred betyeen July 24, 1975 an<J February I, 1982,/ 
. It may serve well a«s documentation Qf an approach that can be used in 
planning for future collabGrative activities involving the saiti^ or 
similap agencies. . 

- •* General, Planning of the Collaborative Studies' 

• \ • h '-met ' ' _ , 

The RCU Executive Committee « at the National Cenper on 1 

July 24-26, 1979, to, develop initial plans for 4 multi-state, multi-year 
programs^ of studies toward the. solution of -substantive issues, problems, 

* , and/or questions in vocational' education. The participants (Appendix A) 
* ' * developed a plan of action which projected the use of combined talents,; 

N ^ funding and in-kind resources of RCU personnel to accomplish research efforts 
that an individual state would be .unabl v e to accomplish alone. 

* Thyee^major ptoblem areas were identified through a systematic review 
of Federal legislative and administrative priorities, recent national 

.Surveys of research needs, arid individual input by participants. The problem 
areas, synopses o^ which are attached (Appendix R) , were: 

1.' Youth Unemployment Education/Wdrk Connection „ 
. ♦ • -(». _.' 

# 2. Validating Vocational Education Effectiveness and Assuring ^ 

Relevance of .^cational Programs 7 

/ 3.' Vocational Education Accessibility to Adults, 



n 



3£ & 



The general plan for research of each problem area included definitive 
statements of 'the problems* or issues, a rationale, some major, objectives 
and expected outcomes. . , . 

It was decided that ^the RCU Executive Committee would seek. self- 
nomination by states to the problem areasy follower} a- planning conference 
at which participants would develop coordinated planO or "projects in each 
problem area. The National Center s would seiSve a catalytic role With 
the RCU Executive committee, ' communicating the plans to the state directors 
of vocational education, convening the problem- area grou'ps for initial * 
planning, and reporting progress at appropriate meetings. 



. Preparing, for ' RCU Involvement - . , 

The president of the National RCU association corresponded' with all- 
RCU directors, describing the planned' act ivit ies and asking for seif-nomination 
•to participate in the efforts' (Appendix C) . The 'executive director of. the 
National Center outlined the projected activities -and -potential roTesTand 
responsibilities in a letter to state- directors of vocational education- 
(Appendix f)>. -Subsequent self-nom^naHons were reviewed ' by the , RCU Executive ' 
Committee at the National Dissemination and Utilization Conference in Columbus 
on October 12, 1919. In, preparation for ^planning conference for the ' , 
participants, the National Center (1) conducted literature searches in each 
problem area, (2) synthesized findings of various studies, (3) arranged fpr 
the participation bf appropriate National Center program staff, and (4) 
"arranged for conference' facilities. *," v ' . . 

i 



Developing Coordinated Plans for 'EaclT s *rogram Area 

• ^ The National RCU Assaciation president described* the'colLaborative studies • 
as "/..the mast unique, challenging,' and exciting .venture ever undertaken by 
the NationalRCU Association.'. .•' iroWEfr opening remarks prepared for the planning 
conference at the National Center on February 12-14," 1980. This 'ioaf erence 
involved representatives ojc self-nominated resWch coordinating -units, key 
professional staff 'from all National Center Divisions*, and member-s of the ' ' 
Dissemination and Utilization Program Staff: ' % ^ 



^--SLtate Research Coordinating , Units 
Arizona v 

. ^ Ken Gabbert' .< 

Florida > - 

, • Margaret Ferqueron ' - 
I Dafce MdOuat 

^'Illinois ' 
Richard °Hof strand 



North-Garoliria 

Fred Manley- 

' Ohio * j 
Bob JBalthaser. 



J 
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Oklahoma ° 
Don Frazier 

Oregon 

Darrell Ward 

Pennsylvania 
Jim Lewis 

South Carolina 
.Maria Sm6ak* 

V 

,West Virginia 
Roy Thomas 

Wisconsin 

Roland Krogstad 
* ¥ Merle Strong 
•* Or v ill-e Nelson" 



A 



National Center • 

Bill Ashley - Dick Miguel 

Juanlta Bice * ' Maria Peter-son 

Bob Darcy % Frank Pratzner 

Ron Denison " Bob Roehrich • 

Marion Johnson Earl Russell / 

J^net Kiplinger - Jill Russell ^ 

Art. Lee Norm Sanger 
Morgan Lewis • * ' Hall Starr 

Aubrey^Lorig Robert Taylor 

Joel Magisos w * • Lloyd Temme 

Floyd McKinaey * Patricia Veasey ■ 

Donna Mertens / > .Cathy W^rmbrod 

~- . • . - Pat Winkfield * • 

ft\e National Center staff , participants- listed above were recommenced 
by the Associate Directors , of Jgg jh National jCenter division as appropriate 

s resource consultants to help.RCU planners address eayh of the Selected . 
problem areas. National Center participants were inyolved^in an,* orient a t Ion . 
session prior to tjie planning conference. „The literature reviews thjx£ ha,d 

- beeiK developed by tjbie National Center staff v in preparation for the conference 
were distributed to "the. RCU participants fornise in the planning 1 sessions. 
The RCU participants at the conference worked with National tenter staff to 
further define the prob^lms areas and to: * ' . t M 

1. Establish objectives for each sturdy, including a refinement of 
* i„ - v previously-develoged goals and fehe creation of specific objectj-verf- 

~^ir % l *" ,v ^? for «each project rphatel... : 



pro j ect iphai^ 0 

^^^^Jescrib^Siie products and outcomes of each project 

# 3, Describe the dissemination strategy for eafih product 

' 4/ Develop a project management plan with*-clearly-def iLngg 
responsibilities '* * 

v 5. Describe the next steps for each project. 

> The agenda of the conference is attached' (Appendix E) . 

**"■ . * 

\ - Sharing Coordinated Plans with the RCUs 

The plan for each problem area -Was presented at the National RCU 
Association conference at/Hilton. Head, South Carolina t on March 12, 1980. 
An opportunity was afforqed to discuss the plans. ,\ ^ 

r • ^Further Planning in. One of the Study A&as 

* *> . 

Subsequent to a discussioji at the RfU Executive Commifcfee meeting 
in Columbus on September 1980, Rational Center staff worked with Oregon 

RCU staf f in Salem to refine the strategy for the effectiveness* prpfrlem 
area* This refinement included a paradigm x>f the study and a program phasing 
plan (figures l f 2, and 3). 



* A series of studies f to assess* Wit effectiveness 0/ vocation*! education and promising program practices. 



EFFECTIVE PEASUWfUT 



Development of instrc*snts to 
ressure outcomes, student status.' 
and program practices. 

1. Synthesize factors/criteria/ 
indicators of outers, 
student status, jnd program 
practices. 

Z. Verify measurement factors/ 
criteria/indicators by 

»- Field Survey • > 

^ b. Panel Review 

y. Pilot test the^instruflvntatlon- 
»nd_rtvl*e r r' 



HEASUREHENt OF EFFECTIVENESS 



Collection, analysis, and 
summarization of -data on the 
effectiveness of vocational 
education. 

I. Oather data about outcofnes. 
student status, and program 
practices. 

Z, Analyze d»te for evidence 
of effectiveness and corre- 
lates 0* effectiveness. 

3. Summrlze, review and 

report flndipqs.* conclu- — 
— stonT7~and recommendations. 

K 



USE OF EFFECTIVENESS DATA 



Publicize program accompli shnents 
Uiroudh official report, "brocnures 
news releasesj and press confer- 
ences. Q , 

^ c 

Recommend program ^improvements 
based upon differences In effec- 
tiveness related to program 
practices. 9 

y$c data as a baselin e 'for future 
assessment of~e7fect"fveneTs^and 
JfltfroveEent. \ 



^ Figure 1. Effectiveness of .Vocational Education 



10/2*769 



11/10/8O 



11/14/RO 
12/04/80 



3/30/81 



Preparation of proqraa Review of description 
description and phasing and phasing plan by: 
Dl'l- Oregon staff 

Arizona staff 
• * Natlorfal Center staff 

(OSU. Evaluation) 



Review and adoption/ 
—♦.adaption of plan by 
HSMY states at O&U 
conference and AVA 
'Convention. 



Contracting process developed 
in- one state to undertake the 
Effectiveness Measurement Study. 



4/1/81 



12/31/81 



Report omtht program 
plan jnd contracting .-i 
effcrt at the RCU 
conference. jf 



3/31/82 



12/3l'82 



/" 



Complete the Effectiveness 
Measurement Study. 

Contracting process developed 
in several states to undertake 
data collection for measure- 
ment of effectiveness. 



Oata col taction 
— a^tarted In several 
States, 



Complete state analysis 
and reporting. 



3/11/33 



6/30/83 



Complete consolidation 
Of state sports into 
national report. 



8*gm use of effectiveness 
data. 




Figure 2. Projected Study Phases 



A t ingle study contracted in 
on* state* onranly an*d *t 
deliver-in? tnstrurientat<cn for 
use m seven) states; -<*<j*it 
involve * panel representative 
of sever*! stmt in verifier* 
tlon of fecton and citer*a. 



S« CONTRACTS j 
WITH APPROPRIATE AGtKCY »- 



Replication of stud/ in several 
states, using comon instrvrren- 
tatton, analysis techniques, and 
reporting formats/ 



$IW£Y$ 



Oregon 



» | Arizona | 
-| Florida | 
- | South Carol ina | • 



Use of consolidated data for national 
purposes and for co^csrisen to indi- 
vidual state data. 

Use 0/ state data for state purposes. 



* REPORTS 



" 1 Arizona 



I Florida 



» | South Carolina J 



Illinois 


Illinois 


I- 








t i- 
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Figure 3. Study Schedule 



The new plan was presented to the Oregon state director of Vocational 
education and his key staff in a special meeting. The new plan received^ 
favorable consideration, but a key concern was whether ^pother* state would 
make a commensurate project commitment, too* In a subsequent, conference 
call with the Arizona RCU, thjLs fact of a'matching commitment was confirmed. 
s Subsequently, the Oregpn RCU was to develop a request *for proposal (Appendix F) 
for the design phase of the effort • The National Center staff was to convene 
other study participants attending the national dissemination and utilization 
corifer^nce to assess the extent of continuing commitment by other RCUs. 



Communicating Progress and jlorvi-fcoring Status 

The National Center reviewed the collaborative study concept with 
participants in a meeting during the t^ational fi&U conference held in, Columbus 
on* November 13, 1980 (Appendix G) / It was 'explained that the multi-year, 
'multi-state efforts would ise comprised of independent studies funded by 
different states, which taken together would solve "a~ larger problem than 
could be undertaken by one state alone. 

The design and plan for. the effectiveness problem area was shared with , 
the group. Reactions to the design and plan were generally favorable;, 
however, a'rfnige of concerns wej,e* expressed, including: 

1, Need for a clearer statement 'of the problem^ definition of 



i 



terms, and statement. of purpose 

Reservations about some earlier work upon which the current 
work might be based ' . 
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3* Need for instrumentation, but . reservations' about data load on 
schools ' . ' * » 

♦ * * 

4* * Concern about audiences for the findings* * ' 

5* Questions about compatibility of definitions between states 

Theresas concensus that Oregon' should proceed with its'design phase. 
Arizona and Florida were prepared to commit resqurces for later phases of 
the study of effect iveness. j 

It was reported that four or * five states were proceeding with the study of 
the alternate delivery systemsjjjsMem ,*jrea . Pennsylvania, and Wisconsin had 
already begun work on the jtfrflclyof the yot^th unemployment problem area. A- * 
PERT had been developed, & state of the art paper was under development, and 
there were plans for a .matrix of exemplars. 



Release of 'Request for Proposal 

v ♦. a . \ ' • - 

, A request for proposal, "Ta Dev&op a Model for .Effectively Measuring 
Secondary Leyel Vocational Education Programs ff\ Oregon, 11 was developed by " * 
the Oregon RCU and Reviewed by National Center, staff (Appendix F) . Based 
upon 'the review, the RFP was revised, cleared and released by the Oregon RCU " 
through the state department of education and released on February 19, 1981. 
Proposals received in response t9 the, RFP were recjeived and reviewed by a panel, 
of representatives from collaborating states and from the National* Center for 
v Research in Vocational Education. The projected schedule qf events from 
release of the RFP. through the proposal review and awatd of -contract is. shown 
in Appendix H. * . ^ . 

— ' ' ' 

Continued Planning During the Sixteenth^ Annual Conference 
' ' : of- the r 

National Research Coordinating Unit Association 

. dfr -April 27 1981, National ^Center staff -sponsored a special meeting of 
participants in mult£-state, multi-year studies. Held during the Sixteenth 
Annual Cbnf erence .of NRCUA, the~meeting was guided by a set of planning 
materials '(Appendix 1) prepared, by the National Center in collaboration with 
the Oregon" RCU. This special meeting involved the following people: 

< rf* D. Balthasar,*OH * " m% 

. Carroll A'. Curtis, TA , - * '* 

Kenneth A*. Gabbert, AZ # ' • * 

Roland Krogstad, WI 

Dave McOuat r PL • ~ 

Greg Morrison, SC 

Peter Seidman, IL * 
' * • -Norm ^ Singer, National Center 
Jay Smink, National Center 
Sheila Stone, OK 

Bill Sto^v^MN*^ - ' 

Eugene Vinarskai,"iQR 
John-Washburn, IL,, 
. - Dee Wilder, JN 
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Marcia Mille?Ti^A&/u'SDE ' . , 

* • . . * Muriel Shay-Taoman| OVktf/USDE 

This meeting .contributed td the progress^.of jnulti-state, multi-year studies 
by* enabling participants to clarify issbes and expectations among collaborators, 
by eliciting new" cotton itments to participate in collaborative studies .and by 
reaffirming the projected'-involvement of RCUs. The meeting, also helped move 
the vocational education ef fectivenfess *study aftiead via particular exchanges of 
critical information through the use of the materials prepared by the National 
Centner to %u±de thee business of the meeting and follow-tfhrbugh activities .' 
During the meeting one state accepted the responsibility of further refining 
the statement of. the research problenj at hand. In general this meeting al-SQ 
afforded ab opportunity for participants in the other studies to compare 
notes and discu^ their progress. 

The Educat ion/Wo rjc Conneotion — Effective Programming for Youth , an 
initial product resulting from the collaborative study concerning youth 
employability, wajs. introduced to RCU personnel and copies of it were distributed 
^by the ^Wisconsin RCU representative. The National Center staff subsequently 
incorporated this product into related .activities of the -National Center's 
Dissemination and Utilization Program. - ' ' • 

♦ • 

Regarding the /Study concerning adult 'accessibility -to vocational education 
in progress among the Florida, West yirginia, and North Carolina" RCUs, a draft 
copy of conceptual .planning materials was* provided to participants for their 
consideration of draft cpncejpjt papers and next steps during meetingSvOf the 
StH^thwide Research. Coordinating Council scheduled for May 1981 and October 1981. 

' Another outcome of this special meeting during the Sixteenth Annual . 
Conference of the .National Research "Coordinating Unit Association was a • 
consensus ion the desirability and utility of engaging in multi-state telephone 
conferences that would be organized, coordinated, and sponsored by'the National 
Center. In cooperation with the Oregon RCU and its contractor; ' plans, were 
set to conduct several conference calls jamorig the RCUs collaborating in the , 
* development of instruments for the ,f ijrst phase of the vocational education 
effectiveness study*, ♦ * 

• : + ■ \ ' ' \ \ , 

* m Telephone Conferencje on^September 22, 1981 x 

In cooperation with the Oregon &CU ; and the Northwest Regional Educational 
Laboratory, contractor to the Oregon RCU, t the National Center scheduled and' 
coordinated* an extended telephone conference ajpong participants in. the "multi- 
state, multi-year study to asse&s vocational education program effectiveness. 
The conference was previewed in a .letter from the Northwest Regional Educational 
Laboratory to participants (Appendix J). Participants included the following:, » 



j 



iL D. Balthasar, OH 
Ken Gabbert, A2 
Dick Hof strand, IL 
Dave McOuat, FL 
Greg Morrison, "SO - 

Marcia Miller, OVAE/USDE \ \ A 

fom Owens, Northwest Lab 

teter Seidman, -IL f * .^J . ' 

Ntfrm Singer, National Center 

Eugene Vinarskai* OR ^ 
8 ^ 



Tour Owens- .of the Northwest Regional Educational Laboratory, the director 
of the iiistrijmentation project under contract to the Oregon* itCU, was 
introduced to the conference call participants. Owens gave a status report 
on the'project Explaining' the process of completing the literature review of 
effectiveness indicators and the measurement of ef fectivenss, Owens acknowl- 
edged 'the reviews, contributions, and suggestions provided by the National 
Center during development of the literature review,, + 
* - , . * * *' o 

v Owens Wd Vinarskai 'further explained that the first meeting of the 
Oregon steering cpmmittee has been held on September 17, 1981 and tW a' 
draft set of effectiveness indicators had -been distributed to participants 
in the^study for thier joint review and recommendations regarding relative 
^importance of the indicators. Owens, previewed next, steps in .conceptualizing 
a model, for measuring effectiveness using a*model developed in South Carolina 
as aji exemplar, ' * , 

The Oregon RCU director previewed the major steps in the project (See 
figurfgfl) and recommended that t an- interim report -be provided during the „ - 
next ftRCUA conference # scheduled for April,' 1^82, Future direcXion of the 
collaborative study will be given consideVatifcfi at that time, as well. 

Several important- topics arose during the telephone conference, . 

One important issue discussed was the need to achieve consensus among the 
participant RCUs regarding techniques for administering the instrument, 

Owens agreed to seek this consensus while eliciting reviews of the draft 
^instrument ajid^through qontinuing discussions among participants in the' 

stt^y.. Another 'important issue concerned ttje* appropriate target audiences to 

whom the instruments should be administered, „ * ~ \ 

<•.-..•.♦•«#• • • - • . - 

The group agreed that a follow-through, telephone conference woulct be y 
arranged ind sponsored by 'the National Center when the Oregon' RCU and North- 
west Lab-stafj:^ required group consideration of instrumentation drafts, 

9m% . % ' : Related Developments During the Fall i981 - . 
" ( Meeting gf the Southwide Research Coordinating Council^ 

During the October 5 thtongh 7, 198L, meeting of the Southwide Research 
Coordinating Council, Dave McOuat of- the Florida RCU reported on the multi- 
state, multi-year study on, alternative approaches to" enhance accessibility 
in* vocational education, Florida, North Carolina, and West Virginia are partici- 
pating in thitf*ef fort , * ■ ' * . 

McOuat shared a draft paper discussing the concepts inherent in this 
collaborative study and requested comments in reaction to the paper. Partici- 
pants envisioned that a common s'urvey < instrument -that can be used by other 
states will be required. Group discussion also led to consensus that this 
multi-state, multi-year project will need to design an approach for describing 
the advantages^ and disadvantages of various strategies to^be considered for 
improving accessbflity tin vocational education. Mc0ua % t sought the participation 
and ext|||pd iftvo.lveraent'Sbf other interested states not yet subscribing to this 

studyv" * * • \ * * x *<*» 1 



' * Reviews by Mail and r Telephone Conference of Instruments 

On October 16,- 1981, Tom Owens of the Northwest Regional Educational 
.Laboratory released to collaborators ijp the vocational education effectiveness 
study the drafts of instruments to be* implemented .later, . Owens requested that 
direct reactions and suggestions for major changes be submitted to him no later 
thfin Friday, October 23, J.9Q1.- 

Owens projected that revision to the instruments based on collaborators* 
* recommendations would be made so that, pilot testing of the instruments qould 
occur beginning on October 26, 1981. 

Subsequently," another- major telephone conference was scheduled and 
planned collaboratively by the' Oregon RCIJ and the Northwest, Regional 
Educational Laboratory, The conference call, sponsored by The National 
Center for Research in Vocational Education and .conducted during a two and 
one-half hour period' onDec ember 10, 1981 included the following persons: 



v 



x % - R: D* Balthasar, 

Ken.Gabbert, AZ 
' ^Dick Hof, strand, IL . * 

Dave McOuat, FL 
. Marcia Miller, OVAE/USDE 
Greg Morrison, SC ( 
Tom Ovfens,. Northwest Lab 
|* Peter Seidman, IL 

Myriel Shay-Tapman, . OVAE/USDE 
Norm Singer, Rational Center 
' ' Jay Smink, National Center » 

, Eugene Vinarskai, OR % 

Aftef a brief orientation and introductions of conference call participants, 
Tom Owens sought particular reacti&ns to the draft instruments he had mailed 
to collaborators. There' was extensive discussion and joint consideration 

of each of the draft instruments during this extended conference. 

* * * - ^ * ' ' .•""*»■ 

During a later phase of the group discussion, Owens turned attention to 
the vocational «education effectiveness model itself. Individual RCU ' i 
representatives commented on aspects of the niodel and how it might be 
perceived and applied in their respective states. - ) ■ *• 

The Oregon RCU direct or s^d^erJJbt^^JUJLs^ a sole source 

» 4 contract with %he Northwest Regional Educational Laboratory r or administering 
the instruments tQ a sample within Oregon* The- Oregon representative asked , 
for confirmations of commitments from other. >states to continue the collaboration 
on this 'multi-state, ynulti-year study so that the benefits of th6 k stmdy might 
accrue to all involved on a nationwide* basis. Other states involved were asked 
to reassess what their^ntinuat ion. role will be. Various RCU representatives 
■ . commented on their continuing commitment as well as particular situations in 

their own states. . 

* * " ' ~* ** 

*. • * 

^ ^ The Oregon RCU director again: previewed the opportunity afforded during 

the next NRCUA conference to present current' progress on this study to the 



other NRCUA colleagues as work proceeds to administer the instruments in the 
states involved. 

* " ' • * * 

Jhe representative from the Office of Vocational and Adult* Education 
suggested that the federal office be contacted to ^arrange further conference 
.calls related to this multi-state, multi-yeafr s'tydy.^* Other participants 
in this telephone conference inquired^ regarding the availability.of support 
from the OVAE for continuation of the collaborative study. 
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Gregory Morrisgn 
Dee Wilder 
Richard Ruff 
• Roland Krogstad 
Will if m Barnes 
Fred Haddad 
Tred Man ley 

v *3o eL*. Magi so s _ 
Juanita Bice ^ 
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Meeting 



Columbus', Ohio; July 24-26, 1979 
* \ Participants 



South Carolina^ 
Tennessee * 
Arizona 

Wisconsin * \ 
Colorado 
Corfhecfticut * - 
North Carolina 

* .* 
National Ceqixer, 
National Cent*' ' 
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Appendix B 



• - * *. * 
Synopsis** of Selected Priority Problems, V 
. % Objectives and Outcornes * : 



Problem I: Youth Unemployment: Education/Work Connection .* ^ 

r ^ Approximately half of the nation's unemployed are under the age 
-of 25. Vocational education needs do expand and restructyre programs to provide 
these individuals' with marketable skills which wi<l I equip them to* enter the work 
force. ^ r A * * • ^ \ 

" Objectives of the proposed* effort would Include (I) to identify 
factors which ^contribute to youth unemployment' in rural and urban areas*} 
C2) to develop strategies for el imihatfng % b^rrlers to youtH participation 
in vocational programs and; (2t) ,to develop alternative* deli very sy^§£ and 
coordination 'mechanisms for Increasing youth participation rates. 

Anticipated outcomes .wou I d fnclude an, analyses of the, barriers 
to youth -participation in vocational skH I tralnmg programs^and-speci f ic 
strategies for the elimination of those identified barriers. Strategies 
wou I d i nc I u3e. the (T) development of alternative delivery systems, (2J 
expanded guidance' and job* development, programs, "(3) cooperative efforts with - 
CETA, business and industry and (4) suggested program modifications to meet 
the needs of youth for skill trailing. M - \ r ' 

^ 1 ° 'I / 

ProbJem 2: .Vafidatina Vocational Educarri&n Effectiveness and Assuring . - 

Re'levance of Vpcational Programs^ - % . • 

As the nation^moves into the '80s, ma jor decisions must be made , 
which will affect the future of Vocational education and it's mission. 
These decisions will -include, legislative enactments, attempts to fund * - 

solution to social problems, as #elL as, the most elective ways to train 
our citfzenry. In making these decisions fe it is jjnperStive that the ' 
effectiveness of vocational education* be determined through a multi-state, 
multi-year study, incorporating such fac"fors as employer/employee* 
satisfaction, economic benefits, job eom^fencies,.and placement. The 
contribution of this study will yield beoef fts 'to Vocational planners, i 
prograrif^developers and evaluators.* - . 

9 * - ' ? 

The objepti ves'of ■'this study wil.l include the development of a 

program plan and procedures for validating the effectiveness of vocational^ * 

education and assuring program relevance.^ Indicators of success- will then 

be' corre I ated to program impact, and delivery strategies developed for 

dec ision-mak^ps at the national, state s^d local levels. 

* * The study will result in the "formulation of strategie^and procedures 
to eval irate vocaTiona I program relevance and effectiveness, Improve . * 
Voc3ticnar educate am I egi s I at ion, assist i n a Uleviati nq socia I probFems 
through the deve-l&pment of viable vocational training progr^rfis', and will 
serve as a. catalyst^ for t the improvement of vocationadV;|Drograms. 



* « 
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Problem 3: Vocational Ed ucation Accessibility to Adults 

♦ — — — — — — ? • — - — 

Barriers currently exist'which^greatly limit the accessibility of 
vocational education -to our qdult citizens. The purpose of this effort Is* 
to address alternative packaging and delivery approaches of vocational 
education yfor adults. The effort is broad fn , scope — emphasizing not only 
retraicung\ and upgrading programming but a I so # suppcrti ve service? needed by 
a'duHts*- , v t - 

* Among the results of this effort will be a catalog of exemplary . • 
delivery systems, a trained cadijq df resource ^ersonnef and the establishment 
of demonstration sites. , * 1 * * 



*A more detailed, statement of the problems, "objections andVoutc6mes 
available* x 
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Appendix C 



— I.l • '| (illl | MM," \\f til* fit 

Iim \ Km*. I , ' tii / lii-.i'itrt'i 



Dear RCU Director: 



September 14 1979 

r 

I. 



Impact, Accountability, "tell ing„thc story", arc .ill co ion terms, frequently 

heard hy vocational educators across' the nation. liVcrxonc is affected by r hew in 
one way or another and. everyone -Is t critHnfc .to. tliOm hi some way, ".ami in mosf -cases . 
their own way -and independently .' » 

, l ... • 

The National RCU Association- l-xccut ivc Committee agreed to rnitintc an effort 
that wiil maximize the use of RUI resources (personnel, lime, and money) , experience 
and capabilities to address some nationa I % issues and problems affecting vocational 
education., Through careful ly coo.rd inatcd planning and the. involvement of .several 
states, all directing the i r resources .toward a comfflSn goal, it will be possible for. 
vocational educators and researchers to demonstrate the impact and effect of 
vocational education on a large Scale. ^ 6 V s 

* * * . * 

In date July the l:xceut«ivc Committee was invited' to the National (filter lo A r 
Rescarcn in Vocational {idueation (NOVl;)] to \\ iscuss the feasibility of 3 coordinated 
mul ti- state/mul td -year project venture for UCIJ/s. The results of the three day 
session ware: a 1 



it 



The identification and scVct ton of three common 'i ssues/proM ems 
affecting -vocational -education reflected in the following topical 
descriptions. ♦ ^ « . 

a. Validating Vocational Education Effectiveness and 
Assuring lU^icyjuiec of Vocational Programs - 

hi- Youth Unemployment: lidtieat i on/Work Conned ion 

c. Vocational hliient ion Accessibility to Adult*. 

An initial -plan for eacllrproject consisting of: " * 



a . Mi ssi on/p rob I em st a t emcnj 

h. I'rtijei;! obji-cl i,vo% 

c. Statement of expected outcomes 



H\. Sec Attachments 1-3. 

#£'-2.- 4 - 

SPv:- ' 



A method for coord iual i nit t be pro rev I plauiiiiir ;md jdmi u f-.ttat ion 
effort. 



.' 15 

1979-1980 




in.: i»» ui.ui, n -rritr i*» 

1 » ■ . 



Page Two 



We think you will want to participate in this vefcuire if you c.an. Carefully 
consider the importance *o£ the effort and the benefits you and vocational education 
can -derive from the project(s). In no way sHfould this effort be considered a 
consortium , whereby money is transferred from one' state to another. Each participant^ 
will "use its own resources and conduct a part of 1 th*e project 'in accordance with the 
project coordination plan.' * . 

Attachment 4 is a form which will aid you in deciding -whether or not^SS* 
• participate in tfie endeavor and provides a means by which you may nominate t your 
state as a participant in this effort. ♦ • V v 

Becayse of the facilitative role of the NGRVE in this/ effort, Dr. Robert 
Tayltfr, Executive Director of NCRVE will be sharing the plans developed during the 
July meeting, with the State Directors 6f Vocational Educationat their national 
conference September 25-27, 1979 in Scottsdale,* Arizona. Dr. Taylor 'will emphasize 
the importance and'value^of these<,cooperative intfer-state efforts and will indicate 
the* resources the NCRVE will offer.' ^ • % ■ . 



licatpv 



I encourage you to discuss this matter with your- State Director before and 



after the state directors con£ 
viable and active participa 
vocational /educators, legis 
willing, and capable of cogp 
regional, or^ national level. 




ence to secure the support necessary t6 make you. 3- 
trust that you will help us demonstrate to ' ♦ 
and the. public, that the'RCU' s»are interested, 
ivp and collaborative effofts at the state, 



Help us, help yourself, help vocaticJnal education. 
We nee'd you. p 



Get involved. • Be 'active. 



SEND YOUR NOMJNATIQN FORM TO ME NO- 
LATER THAN OCTOBER 5/ 1979. 



My address is: 



*Mr. Gregory G. Morrison 
Office of Vocational Education 
1429 Senate^ Street, Room 916 
Columbia, South Carolina 29201 



You 'win note that tfic October 5, 1979 dcadl ine *onl/ ^ives ygu'a few days to 
get your nomination in tjie mail. The Executive Committee is planning tp revjew 
nominations *at a committee meeting immediately following the National Dissemination 
and Utilization Conference in ColumbuvOhio, October 9-11., 1979., . 

If you have questions call me at (803) *58-2358 or any of the n&mbefs of # the 
Executive /Planning Qommittce (list attached). * 4 



v 



Sincerely,, " 



Gregory G-^Morrison, President 
National RCU Association 



Enclosur.cs 



RIC 
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September 26, 1979 




The Executive Committee far the National RCU Association met with the 
] - N&tTb^ in Vocational Education staff to collabora- 

: . : ^tftyel y dgATel qp ai>I^ basis some of the 

^^^bJL^Jf obl^iiis in '>raca ti gna 1 educat* i oii^Av The. . pur poS e *of- .this communication . 
^^^T^M^V^^^A^ doi at^glarini ng^ effort so that you may determine 
;<^e^b^ vemen t of'yoUr vs ta^e -\ 



r 



i Three problems p^maaor concern to vocationgl educators were identi 



iillf ; " T $ the Jpl a hn i ¥ski on V A 'synopsis of the* problems is attached 




f he £l$n^ talents, dollars, and time of RCU and 

WajyohaT^tfriter personnel to wofk toward the sdlution of these problems. 



— * It inconceivable that on^sach of; the three selected problems four .to. 
. six states each^develop projects which fit into a coordinated plan - . 

>The pTari would be .designed to draw on the contribution of each state to 
Solve t'he-larger^problem. ^ 

The process recommended by the planners for identifying and selecting 
.statj&s >to participate was one«of self nomination based on state interest 
. or, current efforts in one^of the problem areas. The State RCU Directors©, • * 
will; be. receiving a communication* from the President of the RCU Assbcia- 
:jtton, Greg Morrison ,^South Carolina. Greg will outline the collaborative 
>^T*nning efforts and* will make the request for self nomination.* 



.e feel, free .to share with me your recommendations' or concerns 
Jj^vV.^r activity. ' '.. 




clbs]ife^ v , * T 
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Appendix E 



The Ohio State University * * 
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L-TH£ NATIONAL CENTER 
^•JMRESEARCH IN VOUTIONAl. EDUCATION 



1960 Kenny Road 
Columbus, Ohio 43210 

Phone: 614-486-3655 

Cable: CTVOCEDOSU/Columbus, Ohio 



Multi-State, Multi-Year Cooperative - » 

Research Efforts with Sta€e \ 
Research- Coordinating Unite: Tactical Planning 
Conference 

February 12-14, 1980 



AGENDA 



? 



1 '*$fc 

Tuesday, February 12 . 
/8:30 .a.m. 
8:45 a.m. i 

9 f s 15 a.m.. \ - ' 

9^30 a.m. 



3jA5 >'.m. 



:r-'€2t<lto* noon- 

ff^{^{? ; -; ; v . , ''V . "/ < 

f^!p\^* - v A :-.J0 p.m.; / 
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196,0 Kenny, Rgom 1-C 
\ J ' - 

• Welcome and Introductions" 

Objectivjes/Procedures/ 
Outcomes of Meeting 

Overview/History of 
Cooperative Effort 



National Center Involvement 
and Commitment* , 

Break 

State Involvement and- Commit- 
ment: Alternative Strategies , 
Constraints and Limitations 

LunchrHRpotf 1-A I 

Introductiqns of RCUs and 
the National Center staff 

Reyifew of Selected Issues// 
Topics ^* 

Procedures for Developing 
Coordination Plans. - " / * 



iV . .. 

IS- 



3:30 p.m. 
4:00 p.m. 
4:30 p.m. 



J 



Identification of /Macro- 
Concerns wa^hiri the Issues 
(small group^lWms 1~B & C) 

Reports of Macro-Concerns 
( to tap*% roup —Room 1-C) 

Interaction with National 
.Center staff. (on -your own) 

Transportation to Stouffer's 
Inn (.OSU van) " , 




*■ Wednesday r February. 13 , 
8:00 'a.nw 

. 8:30 a.m. * . 

> • <* 

8:45 a.m; 

» 

9:45 a.m. ' * 

10:45 a.mj^ 

11:30, a.m. 

v.vX4::00^ noon " V / 
l:00/p.m. - 

'. ■' : , - ■'// 

3: 30- p. m; 
4:30 ^p.m.^ 

\ /: - / 



V.J 



1960 Kenntf, Rooms 1-A/B/C 
■ * — z zrr, 



Transportation to 1960 Kenny 



Review of Pr^c^dures/Charge 
for the Day 

(total 'group — Room 3£-Cf) 



Development of Flow Chart 
q£ Activities 

(small groups — Rooms l^A/B/C) 




Break 

Continue Development of ^ 
Coordination Plans 
(small groups) / s 

Progress Report; By, Groups 
> (total group--Rdom 1-C) 

» • • *\ : 

Lunch - Room 1-A - 



Continue Planning 

(small groups — Rooms 1-A/B/C) 

Progress Report by Groups 
(total group— Room 1-C) 

Transportation to Stouffers 1 
Inn (OSU van) ' 
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29 



-■ma*** 



. Thursday,, February 14- 
7:30 a.m. 



8:30 a.m. 
9: 00 a.m. 

10:45 a.m. 
11:00. a.m. 



Mv . - 

V,-,...,;,.. 

te-' < 



12^K)0 noon' 
1;:00 p.m. 

1: 15, p.m.' 
/ 

2:30 p.m. 
2; 45 p.'m. 




t.. 



T: . •••• , 

&r. x> ^ ■ *v.\ . 1 ^ ' ■ • 



G^qup breakfast with 
Dr.? Robert Taylor — * \ 
Stbuffer's Inn, Canterbury 
Ra6m . , A * 

r" , J * 

; v 

Transportation to 19.60 Ke;my 

Finalize Flow Charts 
.(small groups — Room I-A/B/C) 



XBreak 



7 * 



{ D^ermine Roies arid Re^ponsi- 
) bdlities with Respect, to, ; 
{ Components of Coordination 
Plan 

(small groups) 
* 

iftnch - Room 1-A 



j Travel Forms/Procedures 
(RCU Directors pnly) 

^Critique of Coordination Plans, 
*Ro-Ie, Responsibilities 
\( total" group— Room 1-C) 

&reak , - 

^Refinement of Codr<Jir*a"tion 
Plans * . , • J" 

(Mmall group— Rooms 1-A/B/C) 

"Next Steps" 

(tdfcal .group — Room 1-C) 

Adjournment 



•7 
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||J * .. (Pages 1, 2, 3, 9, 10, 11, 12, 13, 14, 16, and 17 deleted) 



January 12, 3981 



|\v ^0: Secondary' School District Superintendents * 
Secondary Sfchool Principals 
* Secondary School District Directors of 
* • Career/Vocational Education 

Education Service District Superintendents' 
Regional Coordinators of Career/Vocational 
♦ Education 

f* Select Consultants I 



Attached is an RFP for your .information and review entitled: 



* "To Develop a. Model for Effectively Measuring Secondary Level . , 
|pw Vocational Education Programs in Oregon"' 

m'..-- • ■ < . - ■ '. 

life For additional information; contact Eugene T. Vanarskai, Coordinator 
;p|f.;; -of Vocational Education Applied ^Research and 'Exemplary Programs, 

Program Planifing and Evaluation Unit, Oregon Department 6f Education, 
iffr 378-2717 or-the toll f nee number 1--800-452-7813, w • • 



Monty Ku'Haneri . Donald E'/Eg*g.e 



Dfrector * Associate' Superintendent 



|^/; ^Career and Vocational Education ' Instructional^ Services ^ 



REQUEST FOR PROPOSAL 

* /•"- - , 

Notice of Request for Proposal (RFP) "To Develop a Model for Effectively 
Measuring Secondary Level Vocational Education Programs in Oregon" ' 

The purpose of the project to be funded through this^RFP is to develop 
pilot test and deliver instrumentation to be used ur Oregon and several 
other states, .which win* enable*" state agencies to/effectively measure 
secoryiary level vocational education program practices, outcomes when 
correlated with student status. r Also,, to conduct the actual survey of , 
Oregon and prepare a summary report. . 

The project must: 

a. Design, field test instrumentation to enable a state to gather data 
which will effectively measure cJflra document statewide secondary voca- 
tional education program practices and outcomes when correlated with 
student status. • \ 

^ \ ' * ■ . 

b. ActWely involve representation from several states in verification c » 

of effectiveness factors and criteria.^ 

c. Identify practical, effectiveness measurement strategies on secondary 
vocational education programs which are cost effective. 

d. Utilize the effectiveness measurement strategies delineated in con- v 
-junction uith , "implementing and completing a survey of appropriate 

secondary level groups in Oregon. (See Appendix F for a list) 

e. Analyze data and prepare a summary report. « 

In adcjitipn to one above, a through e, proposals need to address matters 
such as: ^ ' 

a. Margin of error which would be acceptable for use during the survey. 

b. . Practical description^ the statistical methods followed in the compila- 

tion of summary^ tables from the primary data. 

4 ^ / * 

c. Distinction between sampling and nonsampl/ing errors. * 

i 

d. Cost of the survey under °such cost centers as development 'work, field 
investigations/testing, analysis, data processf^, report preparation > 
and distribution. 

e- Proyram and student variables to be considered and appropriately 
included in tlie survey of select Oregon groups. - 

The following products must be submitted to the Oregon Department of Educa- 
tion: t . ; , 

a.\ Twenty- five copies of a .tested , efficient model, for effectively 
mea r >urinej secondary level vocational education programs in Oregon, 



V- 



n 



• adaptable or adoptable to a variety of other "State Departments of 
Education. The model must include detailed components such as effec- 
tiveness factors/criteria, instrumentation, implementation/conduct of 
survey, data analysis and cost effectiveness strategies. In addition,' 
a summary report of an actual survey conducted of appropriate groups ' 
in Oregon. THIS DOCUMENT SHQULDSBE SEPARATE FROM THE FINA L REPORT . 

^ — 

* Ten copies of the final report, prepared according to Oregon Department 
of Education special report guidelines.* 



I- 



* See Oregon Department of Education "duidelines for Preparation of Final 
Reports for Special Projects," 11/14/78. 

: - V ^ 
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One contract will be awarded in an amS&nt not' to exceed $20,00Q for ^apfcnfyi- 
mately a seven month project. The award will be announced no later 'than 
April 27, 1981. Project funds cannot be used for purchase* of equipment 
or stipends for att^onding inservice. Indirect costs' may bp requested* irv 
the proposal art he 'Rate Approved* by the Department 6f Education, if such 
an indirect rate is currently, on ftle for the agency; or for a maximum • 
of eight percent .of total project costs. H ^ 

- . - s \ * 

The project may commence as early as May , 198V and continue _up~- through 
November 30, 1981 

The grant award will be based on the content of ttie proposal in keeping 
with the following criteria. .Criteria are cross-referenCfed and weighed^to 
the proposal components in Item 7. # - • ' - \ 

a. The proposal fully and clearly addresses the problem identified in the 
attached "Discussion Paper/'" 

* #~ 

b. The proposal fully and clearly describes the general design of the pro- 
"posed project. The procedures* specify t)ow each goal will be accomplished 
and evaluated. . 

# c. " The goals and/or objectives and activities of the project are clearly ► 
stated and attainable.- . ^ 

d. Strategies and procedures, yhich will provide the required products, are 
clearly described. - *" 

e. /The proposal clearly describes how the nonprofit private^ school (s) will ; 
'have a key role in planning, designing, *and implementing the-model. 

f. Provisions insure that all developed materials are free of bias or 
discrimination based on sex, race, ethnic background arid religion, or 
physical or mental handicap. 

/ 

g. A plan, discussed with the Oregon Department of Education Vocational 
Education^Olssjemi nation 'Special ist*, has been included which describes 
how project results and products will be demonstrated <prior to p/ojec't . . 

'closure. 

h. Participating^ staff were involved in 'th^pla^ning anil" development ; 
of the project proposal. • 

"i . Personnel qualifications^ experience are appropriate for, the pro- 
posed program. '* • " * 

j., '.The budget is appropriate and corresponds to the instructions in Item 
4 on this page of the. HHP. • 

k. • Project evaluation processes are clearly exp^ned. 
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7.,.' The proposal' should iqCltjde t^er^coinponents given below. Letters in 

parentheses refer to the^criteria in Item 6. Ptfint values' ^re assigned 
to' the proposal elements/ * * • , * 



Proposal Components , Point Value 

' • Cover Page * * 0 

" t Body of the proposal, including: 

(1) A description of how fhe/propos#l will- 

specififc^VlJ/ address the problem identified 
v ^ n the RFX^in a minimum number of words', (a) 

V ■ " \ 

■ (2) Project goals arid/or objectives, anq supportive 
activites. (b), (c), (e), (f), and (g) ; , 

' (3) Description of products to be developed. 

^(4) jldefVtlf icatlon of the manner in which the 
* r ' Oregon Department of Education will be involved. 

^(5) A procedure for project evaluation. (k)\ 

• Identification and qualifications "of project staff. 



o' Identification of 'local commitment of staff time, 
* facilities, and related services and support, with 
appropriate budget detailing such commitment, (t) 

• Proposed budget^ including: (j) ^ . 

■ - Personnel salaries and employee benefits. 

- Travel jnd per diem for project director,^ 
consultants, evaluator, etc. / 

- Supplies- 

- Other purchased services, such as technical \ 
assistance sta;ff, substitute or extira tinie t • 

; " .for variou\ staff , evaluator services, ; etc. 
. "* (Explain in detail.) & , 

TOTAL.. ■. 



13 

'27 

15 

. 6< 

5 

15 
10' 

\ 

15 



.100 




/. 



DISCUSSION P/tf'LR. 

. 

. j_ 

A ma*j<*r ffefceurdf qucsj, ion "UhatVis thu effectiveness' of Secondary Level Vocational 
Educat i 6irP7og»:am:Tiri~ Relation to Training and Social Goals?" are the ultimate 
periinete'rs for research to be incorporated into a unique .•umbrella multi-state, 
multi-year project effort. - . - 

The foil owing^cr iter id were u^ed in selecting this research question: 

I.. The need was universal. 

2. ' It was of" significant importance to vocational education. 

3. it lended -itself to the „concept'af a multiple, state 
' and multiple, year effort.! . . , 

{[ . The end product had the potential to serve as a valuable 
' resuurce to % deci sion-makers and influence decisions at 

the federal, regional, and state levels. 

— -V • '• 

Six states, 'in cooperation with the' National Center for Research in Vocational 
, Education, are currently committed to examining this question. The states 
involved are Arizona, Florida, 111 inojs,. Ohio, Oregon and ^South Carolina. 

To dujje the following steps, -related fe^the project, have transpired: 

1. Refining of the research study. < 

Z. Suturing written commitment fur .involvement of above agencies. 

3. Uc-velbiwieii-t of a conceptual model fo£* the project. 

4. ^(JevelopmentHf project phasing plan. • ; 

The purpose or this REP is to secure a Contractor, fok a*single study tojbe 
initially conducted in one state, primarily aimed at developing, fi.eld testing., 
and doh'verin.j project ph.i^.e I instrumentation for use tn several states and 
actually conducting the Mirvc-y of select groups in Oregon. The instrumentation 
Must produce data which will enable a, s^ate to effectively measure and document 
statewide, .secondary vocational .education program .practices, outcomes when 
correlated -with student Uatu^. CSee Appendix' G for conceptual model diagram.) 
Extensive research has already been done on this and related questions by ur.. 
• Robert Itarcy. at the Notional Center for Research in Vocational Education, ip^^ 
Cojuulnr,, Uhio s Consequently, the J.essons and products learned by the Ohto 
Center mu-jt i.e. ihcori>ois»Ui*d~in this project. . 



\ 
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Survey Populatio n 



Total 
Numbers-Statewide 



State jDepartrriervt of Vocational Education Program Specialists 



f& Secondary Leve) -Vocational* Education Caster Instructors' ' - 

\* ilregon State* /JtdvHory Councillor Career and Vocational Education 

« *"** ' * * / * ^ 

^ Secondary Level District VoStional Education Advisory Councils 

x % > / A* * - ^ ~ 

: ^M^ Oregon" Stats University Vocational Education Program Directors/Administrators 
•tp^ 2'are/ts of Vocational -Education CldsteV StudTnts - '•. 




/ 



V 



\ 



parttifiie* instructors; 914 ful 1 tins, instructors 



aRT Includes 

~3ased on l.o X number of\llth/12th grade "students in vocational education 

" - 37 . 
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Appendix G / 




Collaborating States Meeting 
Columbus, Ohio; November 13, 1980 



Participants 



Robert Baithaser 
Carrpll Curtis 
Kenneth Gabbert 
Nancy Hargis 
David 'tycOuat 
Stan Simpson 
Sheila Stone 
Roy Thomas 

Marcia Miller . 

Joel Magisos 
Norman Singer 




Ohio 

Pennsylvania 

Arizona 

Oregon 

Florida > 

Florida 

Oklahoma 

West Virginia 

ED/OVAE 

National Center 
National Center 



J 



r 
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* ■ * * * Appendix H. 



VERNE A. DUNCAN 

STATE SUPERINTENDENT 
OF PLI8UC INSTRUCTION 




W .' ~ STATE OF OREGON 



"DEPARTMENT OF EDUCAIJON 

ffljir \ 1 • " 700 PRINGLE PARKWAY SE ' 

f m> . - • " SALEM, OREGON 97310 



, . ARE£ CODE 503 

J » 378-3569 

\ Toll Free: 1-800^52-7813 



April T6t 1981 



'Norman M. Singer . ' * , 

Senior Research and^ Development Specialist * *. 

The National Center for Research in Vocational Education 

Ohio State University 

I960 Kenny Road ' 

Columbus, OH' 432-10 — 

Dear Norm:. . * 

• "**■•. 

Just a note to thank you for agreeing earlier to serve as a member of 
«je reading panel for. proposal s received in response to our request for 
proposal, memorandum, number 40-1980-81, entitled, "To Develop a Model 
for Measuring Secondary. Level Vocational Wuratinn p™^™ cLL*?..;' . 



^Activity. ' , *-Date . ^ 

Oregon- Department/of Education mai-led RFP's ./ 
toMdentifjedvtarget audience.^- • _ ; ~ February 19, 1981 



j^r '2i .'Postmark: due- date for proposals mailed in — — 

response to the RFP. . x > ',. .' Appil 17, 1981 

f&s\'*'" - ■-• " ■ '' '. ; • ' - " 

m,;,., 3.- Due date for proposals hand delivered in - . 

jg|j • response to the RFP, ; » ' ' . April 20,, *1 981 

Kb- 4. .Copies of proposals received in" response to ' . - 
If tf l e rf ?a along, with a copy of tfie RFP and , 

Pt&^.. •P r PPPsal^iechrjical rev^eW^aterials, are . 

mai led to ^ tHe. reading panel . • April 27* 1981 



Ififkfs. n .;^;nsi--Rai^.inembers- review materials, score 
WMB *#BW;PWaV utilizing a separate 




• May 6, 198,1' 




• ^Rorma/i M. Singer 
VAoril 16, f}81^ 
'Page 2 
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NOTE: It is not wecessary* that you return th£ 
proposals to me, ufftess you desire to do so i 
because of notations, etc. What I actually * 
need is a separate, completed Proposal . • 
Technical Review Material Packet for each 
individual proposal., I would appreciate 
your "returning the review materiajs to me via • 
certified' mail, to help insure athd speed-' delivery. 

Reading panel responsibilities are ^concluded 
at this time. 

Negotiation commences starting witn the agency 
submitting the* highest, numerically ranked 
proposal • 



x 



'> May 6.,^1981 0 



MayJ8, 1981 
May 21, 1981 
-May 25, 1981 



> appreciate your offer to assist, 
call iffyou have que|tibns. 



8r - Contractor selected and notified 
). Notice to MS/MY states, by mail, lofjagency selected. 

Wlien thfe^nlt^rTaT^irrijVes please 

f Best regards, s - 

Eugene" T. Virjarskai , 
Education Program Supervisor 
(503) 378-2717 > , " * 




| 
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Appendix I 



MULTI-STATE/MULTI-YEAR " EFFECTIVENESS" STUDY, 
BREAKFAST MEETING AGENDA 



Thursday , April 2, 7:30 



am 



|; 7:35 am • Norm Singer: 

" The National Center 



*7:40 »am. \ £ugehe Vinarskai: 
Oregon D6 
Education 

7:45' am" * Norm Singer 



^ ^ re 9 0n Department -of 



*• 7:50 am 



Eugene Vinarskai' 



**** 



t, -8:00 art. . Euge'ne Vinarskai and 
Norm Singer 



Mi*** 
9' - 



8:05 am * Muriel Tapman and Marcia • 
„ * Midler: Office of Vocational 
'and Adult Education . * ■ 



Introduction to the limited purpose of the 
meeting 

The RFP is out! ' A quick walk-through of 
highlighted copies 



Orientation ta worksheet: GROUNDWORK FOR 
SURVEY INSTRUMENTATION - 

This worksheet requires information to enable 
the Oregon RCU to contract for a survey 
instrument which serves collaborative needs. 
The worksheet should be completed and given 
to 'Gene Vinarskai as soon as possible — 
preferably this week. ' . ^ 

How will you use .this worksheet? Some 
examples of useful responses . v . 



Orientation, to worksheet:, KEY ISSUES TO 
CONSIDER « 

This worksheet lists issues which need 
immediate joint (^liberation during 
this week''s conference and/or via 
multi -state conference calls very soon. 

Observations, comments 



k?8:10 am 



• Adjourn 



.NOTES 



wax* 
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MULT I -STATE/MULT.I - YEAR "EFFECTIVENESS" STUDY 
^WORKSHEET: KEY ISSUES TO CONSIDER 



\ 



The survey instrument shoulcKbe based on conceptual consistency among 
participating states — to enable useful ^interpretation of data later, v 
As the Oregon RCU starts work with a contractor, you need' to communicate^ 
as much usable information as .possible, regarding WHAT YOUR STATE JUNKS . 
ABOUT VOC ED EFFECTIVENESS AND WHAT IT WANTS TO FIND OUT. ( . ^ . 

The other worksheet enclosed is one-way., How else will you get the 
important messages to Oregon a"t this MSMY study is launched? 



*3. 



This MSMY study will serve a critical need if it derives information 
Congress can .use. What other'audiences might use the findings? - 
What further research hypotheses in your state might be capacitated 
when this MSMY study is completed? Official position^ NRCUA? ^ 

* » 

The RFP Oregon has released details a 6 state involvement in the 

MSMY study. Who's still with us? What does each state commit itself to? 

What expenditures will be involved? Jrfho will bird-dog in your state? 



6. 
7. 

8. 



you prepared for ttie in-kind. collaboration required in Phase One 
MSMY study? Will you gear up soon to conduct the survey later 
(12/31/81) in your state? ' is , 

' •'<"■'■ ■ ■ " 

idation of state results? -(3/3VS3) Who will, do this consolidation? . 
A special consultant tea^rp^id in part by ekch participating state?- 
A contractor to one st 





When can. other states get involved? -About 12/31/81* 



How can consistelffitanalysis of state survey results be achieved among 
collaborating. states? How can the same "lens" be usad among the " . . 
states 'as each interprets its data?/"-' > , 

The National .Center for Research in Vocational Education wishes to 
continue in its facilitating role. How might facilitation and 
cogprfination of communications best con^i.nu^ td ser/e your 
naborative RCU> study — your MSMY study? 



t 




9, DETAIL OTHER ISSU 



SU^BELOW: 



V 
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MOLTI ^STATE/MULTI -YEAR "EFFECTIVENESS" STUDY 
WORKSHEET.: GROUNDWORK FOR SURVEY INSTRUMENTATION 



Please give this worksheet priority consideration. Enter as much information as possible this week — during the NRCUA 
Conference and give your sheet(s) to G^e 4/.ina><skai .. Use the second copy to enter later considerations and mail it ' 
directly to: Eugene Vinarskai, Coordinator, Applied Research and Exemplary Programs, Department of Education, 
700 Pringle Parkway, S E; Salem, Oregon 97310 (Telephone: 503-378-2717). . , - ' 



PARTICIPATING STATE: 



NAME AND. ADDRESS OF CONTACT PERSOti: 



INSTRUCTIONS': To design or> select pilot test instrumentation,- a collaborative perspective on effectiveness imeeded 

. right away. Toward this end please identify and describe in-state, activities (research and/or development)' 
v which involve definitions,- assumptions' or implications regarding voc ed effectiveness. 

Key reference : Oregon's RFP, "To Develop/ a Model for Measuring Secondary Level Vocational Education 
Program Effectiveness." See especially pNfl. - 



i 
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NAMES OF ACTIVITIES 

* 


" RDTFF nFQf DTDTTHtflQ 
DKitr. ULoLKl r- 1 lUtlo 

7 


ASSUMPTIONS RE': "EFFECTIVENESS" 


i.tun mtpux far 

, WHO MIGHT BE , 
CONTACTED? ' '. , 


* * 

- • ' -'4 . 




* 
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Appendix J 



Northwest "™ * 




Regional 
Educational ^ 

Laboratory MMii 300 S.W. Sixth Avenue • Portland; Oregon 97204 • Telephone (503) 24d-6800 x 
September 11 , 1981 



Norman ft. Singer 
' Senior Research and Development Specialist: i , 
0}he National Center for Research ' 
in Vocational Education 
. Ohio State* University 
1960 Kenny Road , 
Columbus, OH 4321(5 



Dear Norman: 



As you know, the Northwest Regional EducatioMl Laboratory is working 
under a contract with the Oregon Department W Education to develop, a 
model for measuring. secondary level vocational education program 
effectiveness. We have" recently completed a review of the literature and 
an examination of- existing practices regarding secondary vocational 
education effectiveness *nd have identified numerous indicators of 
effectiveness.- Some of these are indicators of a quality vocational 
education program while others are. indicators of program "outcomes for 
students or society, since there ar£ many-Snore indicators than we can 
teasibly handle in developing prototype evaluation instruments, we-need 
your assistance in helping us establish priorities among them* The 1 s~ 
priorities you and others identify will be used as we draft evaluation 
instruments, f or five groups: current vocational education students, 
I staff-parents, employers and advisory councils. Although our contract 
excludes develpping an instrument for vocational completers because 
existing instruments for that group. already exist, we will use the 
incarnation in developing a model for vocational education effectiveness. 

* . * 

The attached listing of outcomes' and quality program indicators includes 
feedback we have received to date from several members of the RCU 
executive' grpup at their recent meeting in Columbus, Ohio. After, we^have 

,> anal Y?f d ratin ? s f *"« RCU directors and our Project Steering Committee, 
we will- use the information to develop draft instruments for the five ' 
groups. We will mail-these, drafts to you together with a list of the . 

indicators that were omitted in designing the instruments". ' 
At that time we would appreciate your critique of the draft instruments 
and^wiU give you the opportunity to suggest that we add -a particular 
indicator that, received a relatively low mean 'rating if you feel strongly 

. thafc " « critical indicator. We will then take your review arid 
comments* .into consideration* in preparing the final, draft copy of each' 
instrument. These instruments will be pilot-tested and refined based on 
the results, of the pilot test; 5 • T 

I am arranging through Nprra Singer "to have a joint conference call the 
week -of September, ?lst with the six RCU directors Cooperating in this . 

-S ert - ' ^ ^ liCy iMq6S We need your a9vi ^ e on ' durin 9 the conference 
!^?', are ' (1), whether we should include SASvE directors for input into 

%C^Hr^«-L.---' " ." • • • 

»^f^€<-^f,^-/, :• ^*fe^T. .'. ^ 22 . • 



r 



Norman ty. Singer 
September ' 11, 198}. 
Page 2 
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the key*- Vocational outcomes and (2) whether we should add legislators as 
■ an additional group to be suryeyed through an evaluation rnstrument or 
simply include them as key users ol? the evaluation information* In 
addition, we 'can discuss any important outcomes of vocational education 
you feel we may have overlooked on the rating sheets. 



/ 
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" We appreciate" ypur contribution to this project and know that the final 

products' will improved as a result of your .input along the way. If 
* you have any questions* please feel free to >call me at our toll free 
numbe^ 800 7 547-^339. * " . 

. I look forward to receiving your ratings* in the attached self-addressed 
°. ^envelope by September 25th. 

, : Sincerely, * ^ % 




Thomas R. Owens, . \ 
'Associate Director 
Education arrchWork Program 



TROtagr 
j *. Enclosures 



cc: Gene Vinarskai 
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